Bringing Light Into the Dark: A Large-Scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework.

The heterogeneity in recently published knowledge graph embedding models' implementations, training, and evaluation has made fair and thorough comparisons difficult. To assess the reproducibility of previously published results, we re-implemented and evaluated 21 models in the PyKEEN software package. In this paper, we outline which results could be reproduced with their reported hyper-parameters, which could only be reproduced with alternate hyper-parameters, and which could not be reproduced at all, as well as provide insight as to why this might be the case. We then performed a large-scale benchmarking on four datasets with several thousands of experiments and 24,804 GPU hours of computation time. We present insights gained as to best practices, best configurations for each model, and where improvements could be made over previously published best configurations. Our results highlight that the combination of model architecture, training approach, loss function, and the explicit modeling of inverse relations is crucial for a model's performance and is not only determined by its architecture. We provide evidence that several architectures can obtain results competitive to the state of the art when configured carefully. We have made all code, experimental configurations, results, and analyses available at https://github.com/pykeen/pykeen and https://github.com/pykeen/benchmarking.

Mimicking the radiologists' workflow: Estimating pediatric hand bone age with stacked deep neural networks.

Pediatric endocrinologists regularly order radiographs of the left hand to estimate the degree of bone maturation in order to assess their patients for advanced or delayed growth, physical development, and to monitor consecutive therapeutic measures. The reading of such images is a labor-intensive task that requires a lot of experience and is normally performed by highly trained experts like pediatric radiologists. In this paper we build an automated system for pediatric bone age estimation that mimics and accelerates the workflow of the radiologist without breaking it. The complete system is based on two neural network based models: on the one hand a detector network, which identifies the ossification areas, on the other hand gender and region specific regression networks, which estimate the bone age from the detected areas. With a small annotated dataset an ossification area detection network can be trained, which is stable enough to work as part of a multi-stage approach. Furthermore, our system achieves competitive results on the RSNA Pediatric Bone Age Challenge test set with an average error of 4.56 months. In contrast to other approaches, especially purely encoder-based architectures, our two-stage approach provides self-explanatory results. By detecting and evaluating the individual ossification areas, thus simulating the workflow of the Tanner-Whitehouse procedure, the results are interpretable for a radiologist.

STDP-Compatible Approximation of Backpropagation in an Energy-Based Model.

We show that Langevin Markov chain Monte Carlo inference in an energy-based model with latent variables has the property that the early steps of inference, starting from a stationary point, correspond to propagating error gradients into internal layers, similar to backpropagation. The backpropagated error is with respect to output units that have received an outside driving force pushing them away from the stationary point. Backpropagated error gradients correspond to temporal derivatives with respect to the activation of hidden units. These lead to a weight update proportional to the product of the presynaptic firing rate and the temporal rate of change of the postsynaptic firing rate. Simulations and a theoretical argument suggest that this rate-based update rule is consistent with those associated with spike-timing-dependent plasticity. The ideas presented in this article could be an element of a theory for explaining how brains perform credit assignment in deep hierarchies as efficiently as backpropagation does, with neural computation corresponding to both approximate inference in continuous-valued latent variables and error backpropagation, at the same time.

Bounding the bias of contrastive divergence learning.

Optimization based on k-step contrastive divergence (CD) has become a common way to train restricted Boltzmann machines (RBMs). The k-step CD is a biased estimator of the log-likelihood gradient relying on Gibbs sampling. We derive a new upper bound for this bias. Its magnitude depends on k, the number of variables in the RBM, and the maximum change in energy that can be produced by changing a single variable. The last reflects the dependence on the absolute values of the RBM parameters. The magnitude of the bias is also affected by the distance in variation between the modeled distribution and the starting distribution of the Gibbs chain.